Acting in Partially Observable Environments When Achievement of the Goal Cannot be Guaranteed

The problem of planning in partially observable environments can be regarded as a search problem in belief space where beliefs express the collection of states that are deemed possible. In this paper we address the problem that arises when one of the possible states is a dead-end: a state from which the goal cannot be reached. In such situations, no contingent plan exists, and yet, such situations are common when planning with incomplete information. For example, if a robot has to move to a target while sensing if adjacent cells are free or not, and the map is not known, the (hidden) state where the target is totally blocked by occupied cells is possible and is a dead end. Of course, the robot shouldn’t get paralysed in such cases; it should move toward the target and give up only when one such state is not only possible but certain. However, contingent planning doesn’t help in obtaining such behavior, and neither does the introduction of probabilities. A belief state where a dead-end state is possible is itself a dead-end, and both its worst case cost and its expected cost are infinite. One option in such cases is to find contingent plans or policies that maximize ’coverage’, i.e. the set of possible states for which the solution works. This is the approach that we take in this paper, where we extend a recent action-selection mechanism for contingent planning, that uses a translation into classical planning, to work in such settings. We show that such scenarios are common and that the proposed mechanism has other applications as well. In particular, it can be used to deal with dead-end beliefs that do not contain dead-end states, and to generate meaningful, goal-oriented behavior in solvable but complex contingent settings where state-of-the-art contingent